1. Field of the Invention
The present invention relates to an image processing apparatus that processes an image by detecting a motion vector between two images, and a control method therefor, and a storage medium storing a control program therefor.
2. Description of the Related Art
There is a known image processing apparatus that detects a motion vector between two images and performs various types of image processing using the motion vector.
For example, when dynamic image data is processed using a motion vector between two frames, the dynamic image data can be efficiently compressed. Furthermore, processes, such as a pan-tilt detection and subject tracking of a television camera can be performed using a motion vector.
In addition, applications of image processing using a motion vector are advanced in recent years. For example, a motion vector is used to composite two images by superimposing the images in order to reduce noise in images taken under a low illumination condition.
A motion vector is detected by block matching, or a gradient method, for example, and the block matching is used widely in general.
In the block matching, a motion vector is detected according to a correlation between a reference image and a standard image that are attention images. In this case, each of the reference image and the standard image is divided into rectangular blocks (regions) of a predetermined size, and a correlation between the reference image and the standard image is calculated for every block.
The detection of a motion vector according to a conventional block matching will be described hereafter.
FIG. 16A and FIG. 16B are views showing a detection principle of the motion vector according to the conventional block matching.
FIG. 16A shows a standard image 1601 among two images that are subject to the block matching. The standard image concerned is divided into rectangular regions (referred to as standard blocks hereafter) 1602 of a predetermined size. Each of the standard blocks 1602 has pixels that are arranged in a horizontal direction and a vertical direction. FIG. 16B shows the reference image 1603 among the two images.
In the block matching, the reference image 1603 is searched for a reference block that has high correlation with the standard block 1602. A reference block 1604 of the same size as the standard block 1602 is set in the reference image 1603.
A rectangular search range 1605 that is set to be broader than the reference block 1604 in the reference image 1603 is searched for the position of the reference block 1604 that has the highest correlation with the standard block 1602.
At this time, a SAD (Sum of Absolute Difference) value is calculated for every pixel in the search range 1605, and the position where the SAD value becomes minimum is determined as the position of the reference block 1604 that has the highest correlation with the standard block 1602. Then, the difference between the coordinate of the reference block 1604 that was determined by the search and the coordinate of the standard block 1602 shall be a motion vector 1607.
In the above-mentioned block matching, since the correlations between the standard block 1602 and all the reference blocks 1604 set in the search range 1605 are calculated, the arithmetic processing amount for calculating the SAD values unescapably increases.
There is a known method that constitutes a hierarchy for each of the standard image and the reference image using a high resolution image and a low resolution image that is obtained by reducing the high resolution image. The method detects a motion vector for every hierarchy in order to reduce the arithmetic processing amount (see Japanese Laid-Open Patent Publication (Kokai) No. 2009-153093 (JP 2009-153093A)).
On the other hand, when a motion vector is hierarchically detected using the block matching, a motion vector may be misdetected in a region in which an image is flat (low contrast) and a region of a repeating pattern. That is, since the low contrast region has no pattern that is used to calculate the correlation between the reference block and the standard block, the minimum SAD value is not determined clearly.
Moreover, since the SAD value becomes small at the regions that coincide with the period of the repeating pattern, the high correlation regions will appear in the region of the repeating pattern. As a result, even if the minimum SAD value is selected, it may not correspond to a desired motion vector, and the motion vector may be misdetected.
In particular, if the misdetected motion vector is used to align and composite different two images, the alignment accuracy falls and the quality of the composite image deteriorates.
In order to cope with such misdetection, the technique of JP 2009-153093A excepts the low contrast region and the repeating pattern region from the target for detecting a motion vector when detecting one motion vector integrated in the whole image.
On the other hand, Japanese Laid-Open Patent Publication (Kokai) No. 2010-288110 (JP 2010-288110A) discloses a technique that reduces the arithmetic processing amount by minimizing the frequency of the calculation of a motion vector in the high resolution hierarchy when detecting a motion vector hierarchically.
The technique obtains an edge quantity in the standard block in the low resolution hierarchy, and detects a motion vector in the high resolution hierarchy when the edge quantity is more than a predetermined threshold value. On the other hand, when the edge quantity is less than the threshold value, the technique scales the motion vector in the low resolution hierarchy, and substitutes the scaled motion vector for the motion vector in the high resolution hierarchy.
However, since the method disclosed in JP 2009-153093A excepts the low contrast region and the repeating pattern region, the number of motion vectors decreases as the number of the excepted regions increases. Accordingly, the accuracy of the motion vector after integration may fall on the contrary.
Moreover, the method described in JP 2010-288110A is difficult to detect an exact motion vector because the contrast of the standard block becomes low in the low resolution hierarchy where the edge quantity is less than the threshold value.